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Warehouse Scale Computers 
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Google's Global CDN 
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B4: Software Defined inter-Datacenter WAN 





Traffic 


Google 


History of B4 WAN 



1/1/2010 7/1/2010 1/1/2011 7/1/2011 1/1/2012 7/1/2012 1/1/2013 7/1/2013 1/1/2014 






















oogle 

Mixed SDN Deployment 


B4: SDN Architecture 


Data Center 
Network 


1BGP/ISIS to 
remote sites 


C Paxos 
c Paxos Paxos 



Ready to introduce new network function virtualization (NFV) 
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• The only way to get well 

defined control and data plane 
APIs on a routing HW at that 
time was to build it ourselves 

o Built from merchant silicon 
o OpenFlow support 
o Does not have all features 
o Multiple chassis per site 
o Fully centralized software 
controlled 


B4: SDN Equipment 
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Why SDN? 


• SDN # Cheap Hardware 

• SDN = programmatic decomposition of control, data and 
management planes 

• Well defined APIs => fundamentally easier operational 
model 

• Separation of control and data planes => much higher 
uptime 

• Network function virtualization => new functions rolled out 
in days (vs years) 
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Why SDN? 


Virtual Network 

<=» 

Physical Network 





oogle 

Layer-cake Network 



• Heterogeneous control plane 

• Heterogeneous network apps 

• Large inefficiencies 


Many Networks —► One Network 

Software Defined Network 



* Common network OS 

* Common network apps 

* Global view of network states 
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Anatomy of a Software Defined Network 



Management Plane 


Optical ■ Control Plane 

Restoration 


OpenFlow/PCE-P/... 

\ 


Data Plane 
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Anatomy of a Software Defined Network 



Analytics Management Plane 


Optical ■ Control Plane 

Restoration 


OpenFlow/PCE-P/... 


Data Plane 
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Software Defined Network Configuration 



Transport Protocol 
[ssh, https,..] 




















oogle Towards Declarative Transactional Semantics 

• Good progress in control plane -> dataplane APIs and 
protocols (OpenFlow, PCE-P..) 

• Limited progress in management plane -> control plane 
protocols and APIs 

o Netconf ( RFC 6241 ) is promising, need universal adoption 

• Very limited progress in standard network data model 
definition 

o YANG as modeling language is promising 

o No vendor-neutral data model yet to describe network/device 
configuration 

o No standard network topology model 

• No progress in streaming transfer of bulk-variable/data 

o SNMP is clunky and not that simple © 
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Towards a Common Network Model 


• Network Config model to describe declarative 
configuration 

o Google is working on a rich vendor-neutral network data 
model described in YANG 

• Network Topology model to describe multi-layer network 
topology (Layer-0 - 7) 

o Google made significant progress in structured hierarchical 
description of multi-layer connected graphs using protocol 
buffers * (aka protobuf) 

• We welcome collaboration in developing 
common config and topology models as the 
basis of true software defined network operation 


* http://code.aooqle.eom/p/protobuf/ 





SDN: Beyond the Network Boundaries 
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• Goal 

o Exchange traffic optimally between provider networks (ASNs) 

• Limitations today 

o Mutual intents of traffic exchange are expressed via BGP as *hints* 
o Suboptimal traffic exchange as the peer networks *guess* optimality 

• SDN advantage 

o A common network model and a rich pub/sub API, leveraging cloud 
o Declarative intent expressed by an ISP: 

■ e.g. deliver 10.20.30.0/24 to Denver, 10.20.31.0/24 to Son Francisci 

do_not_deliver traffic in {Portland, Los Angeles}, avoid_congestion in topology_A, 
use augmented_topology_B 


Google 


SDN: Beyond the Network Boundaries 


We welcome collaboration with the 
ISPs in developing programmatic 
traffic exchange 
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Questions? 

bkoley@google.com 



